Iterative immunostaining combined with expansion microscopy and image processing reveals nanoscopic network organization of nuclear lamina.

Investigation of nuclear lamina architecture relies on superresolved microscopy. However, epitope accessibility, labeling density, and detection precision of individual molecules pose challenges within the molecularly crowded nucleus. We developed iterative indirect immunofluorescence (IT-IF) staining approach combined with expansion microscopy (ExM) and structured illumination microscopy to improve superresolution microscopy of subnuclear nanostructures like lamins. We prove that ExM is applicable in analyzing highly compacted nuclear multiprotein complexes such as viral capsids and provide technical improvements to ExM method including three-dimensional-printed gel casting equipment. We show that in comparison with conventional immunostaining, IT-IF results in a higher signal-to-background ratio and a mean fluorescence intensity by improving the labeling density. Moreover, we present a signal-processing pipeline for noise estimation, denoising, and deblurring to aid in quantitative image analyses and provide this platform for the microscopy imaging community. Finally, we show the potential of signal-resolved IT-IF in quantitative superresolution ExM imaging of nuclear lamina and reveal nanoscopic details of the lamin network organization-a prerequisite for studying intranuclear structural coregulation of cell function and fate.

Ring artifact and Poisson noise attenuation via volumetric multiscale nonlocal collaborative filtering of spatially correlated noise.

X-ray micro-tomography systems often suffer from high levels of noise. In particular, severe ring artifacts are common in reconstructed images, caused by defects in the detector, calibration errors, and fluctuations producing streak noise in the raw sinogram data. Furthermore, the projections commonly contain high levels of Poissonian noise arising from the photon-counting detector. This work presents a 3-D multiscale framework for streak attenuation through a purposely designed collaborative filtering of correlated noise in volumetric data. A distinct multiscale denoising step for attenuation of the Poissonian noise is further proposed. By utilizing the volumetric structure of the projection data, the proposed fully automatic procedure offers improved feature preservation compared with 2-D denoising and avoids artifacts which arise from individual filtering of sinograms.

Feed-forward neural network as nonlinear dynamics integrator for supercontinuum generation: erratum.

We present an erratum to our Letter [Opt. Lett.47, 802 (2022)10.1364/OL.448571]. This erratum corrects an error in the sign of one of the higher-order dispersion coefficient used in the simulations of Figs. 2 and 4, as well as in Figs. S1 and S3. The simulations in the original Letter were performed using the correct value, and therefore this correction does not affect any of the results and conclusions of the original Letter.

Feed-forward neural network as nonlinear dynamics integrator for supercontinuum generation.

The nonlinear propagation of ultrashort pulses in optical fibers depends sensitively on the input pulse and fiber parameters. As a result, the optimization of propagation for specific applications generally requires time-consuming simulations based on the sequential integration of the generalized nonlinear Schrödinger equation (GNLSE). Here, we train a feed-forward neural network to learn the differential propagation dynamics of the GNLSE, allowing emulation of direct numerical integration of fiber propagation, and particularly the highly complex case of supercontinuum generation. Comparison with a recurrent neural network shows that the feed-forward approach yields faster training and computation, and reduced memory requirements. The approach is generic and can be extended to other physical systems.

Ring artifact reduction via multiscale nonlocal collaborative filtering of spatially correlated noise.

X-ray micro-tomography systems often suffer severe ring artifacts in reconstructed images. These artifacts are caused by defects in the detector, calibration errors, and fluctuations producing streak noise in the raw sinogram data. In this work, these streaks are modeled in the sinogram domain as additive stationary correlated noise upon logarithmic transformation. Based on this model, a streak removal procedure is proposed where the Block-Matching and 3-D (BM3D) filtering algorithm is applied across multiple scales, achieving state-of-the-art performance in both real and simulated data. Specifically, the proposed fully automatic procedure allows for attenuation of streak noise and the corresponding ring artifacts without creating major distortions common to other streak removal algorithms.

Anisotropic Denoising of 3D Point Clouds by Aggregation of Multiple Surface-Adaptive Estimates.

3D point clouds commonly contain positional errors which can be regarded as noise. We propose a point cloud denoising algorithm based on aggregation of multiple anisotropic estimates computed on local coordinate systems. These local estimates are adaptive to the shape of the surface underlying the point cloud, leveraging an extension of the Local Polynomial Approximation (LPA) - Intersection of Confidence Intervals (ICI) technique to 3D point clouds. The adaptivity due to LPA-ICI is further strengthened by the dense aggregation with data-driven weights. Experimental results demonstrate state-of-the-art restoration quality of both sharp features and smooth areas.

Noise modeling and variance stabilization of a computed radiography (CR) mammography system subject to fixed-pattern noise.

In this work we model the noise properties of a computed radiography (CR) mammography system by adding an extra degree of freedom to a well-established noise model, and derive a variance-stabilizing transform (VST) to convert the signal-dependent noise into approximately signal-independent. The proposed model relies on a quadratic variance function, which considers fixed-pattern (structural), quantum and electronic noise. It also accounts for the spatial-dependency of the noise by assuming a space-variant quantum coefficient. The proposed noise model was compared against two alternative models commonly found in the literature. The first alternative model ignores the spatial-variability of the quantum noise, and the second model assumes negligible structural noise. We also derive a VST to convert noisy observations contaminated by the proposed noise model into observations with approximately Gaussian noise and constant variance equals to one. Finally, we estimated a look-up table that can be used as an inverse transform in denoising applications. A phantom study was conducted to validate the noise model, VST and inverse VST. The results show that the space-variant signal-dependent quadratic noise model is appropriate to describe noise in this CR mammography system (errors< 2.0% in terms of signal-to-noise ratio). The two alternative noise models were outperformed by the proposed model (errors as high as 14.7% and 9.4%). The designed VST was able to stabilize the noise so that it has variance approximately equal to one (errors< 4.1%), while the two alternative models achieved errors as high as 26.9% and 18.0%, respectively. Finally, the proposed inverse transform was capable of returning the signal to the original signal range with virtually no bias.

Collaborative Filtering of Correlated Noise: Exact Transform-Domain Variance for Improved Shrinkage and Patch Matching.

Collaborative filters perform denoising through transform-domain shrinkage of a group of similar patches extracted from an image. Existing collaborative filters of stationary correlated noise have all used simple approximations of the transform noise power spectrum adopted from methods which do not employ patch grouping and instead operate on a single patch. We note the inaccuracies of these approximations and introduce a method for the exact computation of the noise power spectrum. Unlike earlier methods, the calculated noise variances are exact even when noise in one patch is correlated with noise in any of the other patches. We discuss the adoption of the exact noise power spectrum within shrinkage, in similarity testing (patch matching), and in aggregation. We also introduce effective approximations of the spectrum for faster computation. Extensive experiments support the proposed method over earlier crude approximations used by image denoising filters such as Block-Matching and 3D-filtering (BM3D), demonstrating dramatic improvement in many challenging conditions.

Technical Note: Noise models for virtual clinical trials of digital breast tomosynthesis.

PURPOSE: To investigate the use of an affine-variance noise model, with correlated quantum noise and spatially dependent quantum gain, for the simulation of noise in virtual clinical trials (VCT) of digital breast tomosynthesis (DBT). METHODS: Two distinct technologies were considered: an amorphous-selenium (a-Se) detector with direct conversion and a thallium-doped cesium iodide (CsI(Tl)) detector with indirect conversion. A VCT framework was used to generate noise-free projections of a uniform three-dimensional simulated phantom, whose geometry and absorption match those of a polymethyl methacrylate (PMMA) uniform physical phantom. The noise model was then used to generate noisy observations from the simulated noise-free data, while two clinically available DBT units were used to acquire projections of the PMMA physical phantom. Real and simulated projections were then compared using the signal-to-noise ratio (SNR) and normalized noise power spectrum (NNPS). RESULTS: Simulated images reported errors smaller than 4.4% and 7.0% in terms of SNR and NNPS, respectively. These errors are within the expected variation between two clinical units of the same model. The errors increase to 65.8% if uncorrelated models are adopted for the simulation of systems featuring indirect detection. The assumption of spatially independent quantum gain generates errors of 11.2%. CONCLUSIONS: The investigated noise model can be used to accurately reproduce the noise found in clinical DBT. The assumption of uncorrelated noise may be adopted if the system features a direct detector with minimal pixel crosstalk.

Method for Simulating Dose Reduction in Digital Breast Tomosynthesis.

This paper proposes a new method of simulating dose reduction in digital breast tomosynthesis, starting from a clinical image acquired with a standard radiation dose. It considers both signal-dependent quantum and signal-independent electronic noise. Furthermore, the method accounts for pixel crosstalk, which causes the noise to be frequency-dependent, thus increasing the simulation accuracy. For an objective assessment, simulated and real images were compared in terms of noise standard deviation, signal-to-noise ratio (SNR) and normalized noise power spectrum (NNPS). A two-alternative forced-choice (2-AFC) study investigated the similarity between the noise strength of low-dose simulated and real images. Six experienced medical physics specialists participated on the study, with a total of 2 160 readings. Objective assessment showed no relevant trends with the simulated noise. The relative error in the standard deviation of the simulated noise was less than 2% for every projection angle. The relative error of the SNR was less than 1.5%, and the NNPS of the simulated images had errors less than 2.5%. The 2-AFC human observer experiment yielded no statistically significant difference ( =0.84) in the perceived noise strength between simulated and real images. Furthermore, the observer study also allowed the estimation of a dose difference at which the observer perceived a just-noticeable difference (JND) in noise levels. The estimated JND value indicated that a change of 17% in the current-time product was sufficient to cause a noticeable difference in noise levels. The observed high accuracy, along with the flexible calibration, make this method an attractive tool for clinical image-based simulations of dose reduction.

Difference of Gaussians revolved along elliptical paths for ultrasound fetal head segmentation.

We present a fully automatic method to segment the skull from 2-D ultrasound images of the fetal head and to compute the standard biometric measurements derived from the segmented images. The method is based on the minimization of a novel cost function. The cost function is formulated assuming that the fetal skull has an approximately elliptical shape in the image and that pixel values within the skull are on average higher than in surrounding tissues. The main idea is to construct a template image of the fetal skull parametrized by the ellipse parameters and the calvarial thickness. The cost function evaluates the match between the template image and the observed ultrasound image. The optimum solution that minimizes the cost is found by using a global multiscale, multistart Nelder-Mead algorithm. The method was qualitatively and quantitatively evaluated using 90 ultrasound images from a recent segmentation grand challenge. These images have been manually analyzed by three independent experts. The segmentation accuracy of the automatic method was similar to the inter-expert segmentation variability. The automatically derived biometric measurements were as accurate as the manual measurements. Moreover, the segmentation accuracy of the presented method was superior to the accuracy of the other automatic methods that have previously been evaluated using the same data.

Noise parameter mismatch in variance stabilization, with an application to Poisson-Gaussian noise estimation.

In digital imaging, there is often a need to produce estimates of the parameters that define the chosen noise model. We investigate how the mismatch between the estimated and true parameter values affects the stabilization of variance of signal-dependent noise. As a practical application of the general theoretical considerations, we devise a novel approach for estimating Poisson­Gaussian noise parameters from a single image, combining variance-stabilization and noise estimation for additive Gaussian noise. Furthermore, we construct a simple algorithm implementing the devised approach. We observe that when combined with optimized rational variance-stabilizing transformations, the algorithm produces results that are competitive with those of a state-of-the-art Poisson­Gaussian estimator.

Joint removal of random and fixed-pattern noise through spatiotemporal video filtering.

We propose a framework for the denoising of videos jointly corrupted by spatially correlated (i.e., nonwhite) random noise and spatially correlated fixed-pattern noise. Our approach is based on motion-compensated 3D spatiotemporal volumes, i.e., a sequence of 2D square patches extracted along the motion trajectories of the noisy video. First, the spatial and temporal correlations within each volume are leveraged to sparsify the data in 3D spatiotemporal transform domain, and then the coefficients of the 3D volume spectrum are shrunk using an adaptive 3D threshold array. Such array depends on the particular motion trajectory of the volume, the individual power spectral densities of the random and fixed-pattern noise, and also the noise variances which are adaptively estimated in transform domain. Experimental results on both synthetically corrupted data and real infrared videos demonstrate a superior suppression of the random and fixed-pattern noise from both an objective and a subjective point of view.

Indirect estimation of signal-dependent noise with nonadaptive heterogeneous samples.

We consider the estimation of signal-dependent noise from a single image. Unlike conventional algorithms that build a scatterplot of local mean-variance pairs from either small or adaptively selected homogeneous data samples, our proposed approach relies on arbitrarily large patches of heterogeneous data extracted at random from the image. We demonstrate the feasibility of our approach through an extensive theoretical analysis based on mixture of Gaussian distributions. A prototype algorithm is also developed in order to validate the approach on simulated data as well as on real camera raw images.

Evaluation and comparison of current fetal ultrasound image segmentation methods for biometric measurements: a grand challenge.

This paper presents the evaluation results of the methods submitted to Challenge US: Biometric Measurements from Fetal Ultrasound Images, a segmentation challenge held at the IEEE International Symposium on Biomedical Imaging 2012. The challenge was set to compare and evaluate current fetal ultrasound image segmentation methods. It consisted of automatically segmenting fetal anatomical structures to measure standard obstetric biometric parameters, from 2D fetal ultrasound images taken on fetuses at different gestational ages (21 weeks, 28 weeks, and 33 weeks) and with varying image quality to reflect data encountered in real clinical environments. Four independent sub-challenges were proposed, according to the objects of interest measured in clinical practice: abdomen, head, femur, and whole fetus. Five teams participated in the head sub-challenge and two teams in the femur sub-challenge, including one team who tackled both. Nobody attempted the abdomen and whole fetus sub-challenges. The challenge goals were two-fold and the participants were asked to submit the segmentation results as well as the measurements derived from the segmented objects. Extensive quantitative (region-based, distance-based, and Bland-Altman measurements) and qualitative evaluation was performed to compare the results from a representative selection of current methods submitted to the challenge. Several experts (three for the head sub-challenge and two for the femur sub-challenge), with different degrees of expertise, manually delineated the objects of interest to define the ground truth used within the evaluation framework. For the head sub-challenge, several groups produced results that could be potentially used in clinical settings, with comparable performance to manual delineations. The femur sub-challenge had inferior performance to the head sub-challenge due to the fact that it is a harder segmentation problem and that the techniques presented relied more on the femur's appearance.

Nonlocal transform-domain filter for volumetric data denoising and reconstruction.

We present an extension of the BM3D filter to volumetric data. The proposed algorithm, BM4D, implements the grouping and collaborative filtering paradigm, where mutually similar d-dimensional patches are stacked together in a (d+1)-dimensional array and jointly filtered in transform domain. While in BM3D the basic data patches are blocks of pixels, in BM4D we utilize cubes of voxels, which are stacked into a 4-D "group." The 4-D transform applied on the group simultaneously exploits the local correlation present among voxels in each cube and the nonlocal correlation between the corresponding voxels of different cubes. Thus, the spectrum of the group is highly sparse, leading to very effective separation of signal and noise through coefficient shrinkage. After inverse transformation, we obtain estimates of each grouped cube, which are then adaptively aggregated at their original locations. We evaluate the algorithm on denoising of volumetric data corrupted by Gaussian and Rician noise, as well as on reconstruction of volumetric phantom data with non-zero phase from noisy and incomplete Fourier-domain (k-space) measurements. Experimental results demonstrate the state-of-the-art denoising performance of BM4D, and its effectiveness when exploited as a regularizer in volumetric data reconstruction.

Optimal inversion of the generalized Anscombe transformation for Poisson-Gaussian noise.

Many digital imaging devices operate by successive photon-to-electron, electron-to-voltage, and voltage-to-digit conversions. These processes are subject to various signal-dependent errors, which are typically modeled as Poisson-Gaussian noise. The removal of such noise can be effected indirectly by applying a variance-stabilizing transformation (VST) to the noisy data, denoising the stabilized data with a Gaussian denoising algorithm, and finally applying an inverse VST to the denoised data. The generalized Anscombe transformation (GAT) is often used for variance stabilization, but its unbiased inverse transformation has not been rigorously studied in the past. We introduce the exact unbiased inverse of the GAT and show that it plays an integral part in ensuring accurate denoising results. We demonstrate that this exact inverse leads to state-of-the-art results without any notable increase in the computational complexity compared to the other inverses. We also show that this inverse is optimal in the sense that it can be interpreted as a maximum likelihood inverse. Moreover, we thoroughly analyze the behavior of the proposed inverse, which also enables us to derive a closed-form approximation for it. This paper generalizes our work on the exact unbiased inverse of the Anscombe transformation, which we have presented earlier for the removal of pure Poisson noise.

Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms.

We propose a powerful video filtering algorithm that exploits temporal and spatial redundancy characterizing natural video sequences. The algorithm implements the paradigm of nonlocal grouping and collaborative filtering, where a higher dimensional transform-domain representation of the observations is leveraged to enforce sparsity, and thus regularize the data: 3-D spatiotemporal volumes are constructed by tracking blocks along trajectories defined by the motion vectors. Mutually similar volumes are then grouped together by stacking them along an additional fourth dimension, thus producing a 4-D structure, termed group, where different types of data correlation exist along the different dimensions: local correlation along the two dimensions of the blocks, temporal correlation along the motion trajectories, and nonlocal spatial correlation (i.e., self-similarity) along the fourth dimension of the group. Collaborative filtering is then realized by transforming each group through a decorrelating 4-D separable transform and then by shrinkage and inverse transformation. In this way, the collaborative filtering provides estimates for each volume stacked in the group, which are then returned and adaptively aggregated to their original positions in the video. The proposed filtering procedure addresses several video processing applications, such as denoising, deblocking, and enhancement of both grayscale and color data. Experimental results prove the effectiveness of our method in terms of both subjective and objective visual quality, and show that it outperforms the state of the art in video denoising.

Modeling the performance of image restoration from motion blur.

When dealing with motion blur there is an inevitable trade-off between the amount of blur and the amount of noise in the acquired images. The effectiveness of any restoration algorithm typically depends on these amounts, and it is difficult to find their best balance in order to ease the restoration task. To face this problem, we provide a methodology for deriving a statistical model of the restoration performance of a given deblurring algorithm in case of arbitrary motion. Each restoration-error model allows us to investigate how the restoration performance of the corresponding algorithm varies as the blur due to motion develops. Our modeling treats the point-spread-function trajectories as random processes and, following a Monte-Carlo approach, expresses the restoration performance as the expectation of the restoration error conditioned on some motion-randomness descriptors and on the exposure time. This allows to coherently encompass various imaging scenarios, including camera shake and uniform (rectilinear) motion, and, for each of these, identify the specific exposure time that maximizes the image quality after deblurring.

A closed-form approximation of the exact unbiased inverse of the Anscombe variance-stabilizing transformation.

We presented an exact unbiased inverse of the Anscombe variance-stabilizing transformation in [M. Mäkitalo and A. Foi, "Optimal inversion of the Anscombe transformation in low-count Poisson image denoising," IEEE Trans. Image Process., vol. 20, no. 1, pp. 99-109, Jan. 2011.] and showed that when applied to Poisson image denoising, the combination of variance stabilization and state-of-the-art Gaussian denoising algorithms is competitive with some of the best Poisson denoising algorithms. We also provided a MATLAB implementation of our method, where the exact unbiased inverse transformation appears in nonanalytical form. Here, we propose a closed-form approximation of the exact unbiased inverse in order to facilitate the use of this inverse. The proposed approximation produces results equivalent to those obtained with the accurate (nonanalytical) exact unbiased inverse, and thus, notably better than one would get with the asymptotically unbiased inverse transformation that is commonly used in applications.